Using Som and Lvq for Hmm Training 1.1.1 the Segmental Som Training

1.1 New training methods for the HMMs The training of the context-independent phoneme models for a minimal recognition error rate is diicult, because the variability of the phonemes in diierent conditions and contexts is substantial and the output densities of diierent phonemes do also overlap. A structure that can automatically adapt to all the complicated density functions, has a vast number of parameters and for proper estimation, the quality and quantity of the available training data is crucial. The size of the models and the training database demand robustness to the initial parameter values in order to avoid an excessively large number of training epochs and long training times. The problem in practice with the widely spread training algorithms such as the segmental K-means (SKM) 8]and the segmental Generalized Probabilistic Descent (SGPD) 1]is that they sometimes converge slowly to low error rates unless good initial models are available. Several common initialization methods have been compared for the mixture density hidden Markov models (MDHMM). The best results in terms of quickly obtained low nal error rates in the automatic speech recognition (ASR) tests were obtained by using the Self-Organizing Maps (SOM) 2]to rst train phoneme dependent code-books and then use the codebook vectors as kernel centroids for the mixture densities. If the Learning Vector quantization (LVQ) 2]is used in the training after the SOMs, small improvements in the initialization can be achieved, but the SOM training can be performed much faster, because each phoneme codebook can be individually trained as a small SOM. The developed segmental SOM training for the HMMs 5]resembles to the conventional SKM type Viterbi training, but the main diierence is that the parameters of mixtures belonging to the neighborhood of the best-matching component are also adapted. The motivation for the neighborhood adaptation is the parameter smoothing, where the level of the smoothing compared to the tting accuracy to the training data is controlled by the neighborhood size. A wide neighborhood at the beginning ensures also that all the available codebook units will be drawn into useful regions in the input space. Compared to the codebooks trained without smoothing (e.g. by SKM) the accuracy provided by the best-matching Gaussian is usually worse, but that of the next (K ? 1)-best matches will be better, however, providing generalization for slightly discrepant characteristics of the test data. The motivation to have ordered density codebooks is to enable accelerated state pdf …